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1.0  DOCUMENT  PURPOSE  AND  SCOPE 


The  purpose  of  this  document  is  to  describe  the  Phase  II  Training 
Information  Management  System  (TIMS)  evaluation  conducted  at  the  U.S. 
Army  Armor  Center  (Fort  Knox,  Kentucky)  during  the  month  of  November 
1985,  and  to  summarize  the  results  obtained. 

This  document  contains  six  major  sections: 

o  1.0  Purpose  and  Scope 

o  2.0  Background 

o  3.0  Description  of  Evaluation  Process 
o  4.0  Summary  of  Questionnaire  Responses 
o  5.0  Summary  of  BNCOC  Evaluations 
o  6.0  Conclusions  and  Recommendations 
20  BACKGROUND 

The  TIMS  is  a  concept  for  a  computer-based  system  that  can  be  used  by 
Army  personnel  to  collect  and  display  training  evaluation  data  during  field 
training  exercises,  and  to  generate  summary  evaluation  reports  following 
the  training  exercises.  The  TIMS  evaluated  at  Fort  Knox  consisted  of  two 
subsystems:  (1)  The  Electronic  Clipboard  Subsystem  (ECS),  a  hand-held 
portable  device  to  display  and  collect  field  training  and  evaluation  data; 
and  (2)  a  Training  Base  Station  Emulator  (TBSE),  which  configures  the 
hand-held  ECS  for  field  operation  and  summarizes  and  prints  the  collected 
data.  The  Fort  Knox  field  evaluation  of  TIMS  was  designed  to  allow 
assessment  of  users'  opinions  regarding  the  TIMS'  current  hardware  and 
software  capabilities,  and  collection  of  user's  recommendations  for  both 
immediate  and  long-term  product  improvement. 


aO  DESCRIPTION  OF  EVALUATION  PROCESS 


During  the  evaluation,  feedback  was  obtained  from  both  Fort  Knox  Armor 
School  Basic  Non-Commissioned  Officer  Course  (BNCOC) 
instructors/evaluators  and  a  cross  sample  of  instructors  and  evaluators 
from  five  other  USARMC  departments  and  courses  (Training  Group, 
Command  and  Staff,  Weapons,  Maintenance  and  the  NCO  Academy). 

All  subjects  were  familiarized  with  TIMS  operation  before  they  were 
asked  to  evaluate  the  system.  All  subjects  received  a  tutorial  and 
hands-on  "user  familiarization"  orientation  on  the  system.  In  addition, 
some  of  the  BNCOC  instructors/evaluators  also  had  an  opportunity  to  use 
the  system  in  an  operational  environment  prior  to  making  their 
evaluations. 

Structured  questionnaires  were  used  to  obtain  feedback  from  30 
respondents.  Pre-prepared  questions  were  presented  to  TIMS  evaluators 
both  in  written  and  verbal  format  by  an  "evaluation  coordinator".  TIMS 
evaluators  were  asked  to  respond  to  certain  of  the  questions  by  marking  a 
5-point  rating  scale.  Other  questions  were  used  by  the  evaluation 
coordinator  in  order  to  aid  respondents  in  verbalizing  additional  thoughts, 
feelings,  and  recommendations  which  might  not  be  made  explicit  through 
the  more  structured  rating  process.  All  such  "free  form”  verbalizations 
made  by  TIMS  evaluators  were  also  documented.  Questionnaire  responses 
are  summarized  in  section  4.0. 

Due  to  the  tight  time  schedules  of  the  BNCOC  evaluators,  it  was  found 
more  expedient  to  extract  evaluation  data  from  them  in  a  more  free-form 
format,  by  the  use  of  verbal  questions  (modeled  after  questions  on  the 
questionnaire)  and  "role  playing",  whereby  the  evaluators  ran  through 
scenarios  for  using  the  Clipboard  under  operational  conditions  and 
reported  their  thoughts  and  impressions  as  to  the  strengths  and 
weaknesses  of  the  current  design.  There  were  also  BNCOC  evaluators  who 
used  the  Clipboard  during  Gunnery  and  Land  Navigation  exercises,  and  under 
day  and  night  conditions.  BNCOC  evaluator  responses  are  summarized  in 
section  5.0. 


4.0  SUMMARY  OF  QUESTTONNAJRE  RESPONSES 


TIMS  evaluation  information  was  obtained  from  thirty  evaluators  using 
structured  questionnaires  (see  Appendix  A  for  an  example  questionnaire). 
Pre-prepared  questions  were  presented  to  TIMS  evaluators  both  in  written 
and  verbal  format  by  an  "evaluation  coordinator".  TIMS  evaluators  were 
asked  to  respond  to  certain  questions  by  marking  a  5-point  rating  scale. 
Other  questions  were  used  by  the  evaluation  coordinator  in  order  to  aid 
respondents  in  verbalizing  additional  thoughts,  feelings,  and 
recommendations.  All  such  "free  form"  verbalizations  were  also 
documented  and  used  in  the  subsequent  analysis. 

Most  respondents  answered  all  of  the  questions.  As  a  rule,  questions  were 
only  left  unanswered  where  the  respondents  did  not  have  an  opportunity  to 
use  the  TIMS  under  the  specific  conditions  specified  in  the  question. 

Prior  to  filling  out  the  TIMS  evaluation  questionnare,  respondents  were 
asked  to  fill  out  a  short  "Profile  Questionnaire"  indicating  background 
information  such  as  their  time  in  the  service,  current  duty  assignment, 
and  previous  experience  with  computers.  Analysis  of  the  Profile 
Questionnaires  revealed  that  all  but  four  respondents  were  E-6s  or  E-7s, 
and  overall  their  previous  experience  with  computers  was  limited. 

In  the  following  pages  the  results  from  the  questionnaire  evaluation  are 
reported.  Results  are  summarized  and  reported  for  each  questionnaire 
evaluation  item  as  follows.  For  each  item,  the  evaluation  item  itself  is 
listed,  followed  by  a  summary  of  the  number  of  individuals  responding  to 
the  particular  item  and  the  distribution  of  their  responses  on  the  five- 
point  rating  scale  (i.e.,  number  and  percent  of  respondents  giving  each 
possible  rating  response).  An  interpretation  of  the  result  for  the  item  is 
then  made  based  upon  consideration  of  the  distribution  of  rating  scale 
responses  and  upon  consideration  of  the  free-form  responses,  opinions, 
and  recommendations  expressed  by  respondents  (and  also  recorded  on  the 
questionnaires)  during  the  evaluation. 
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Clipboard  Physical  Configuration  Questions 


4.1  SIZE 

4.1.1  Questionnaire  Evaluation  Item: 

*  The  overall  size  (length,  width,  and  thickness)  was  appropriate  for 
me  to  use  the  Clipboard. 

4.1.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

40% 

10% 

50% 

Respondents  per  category 

12 

3 

15 

0 

0 

Too  Just  Too 

Big  Right  Small 


4.1 .3  Interpretation: 

*  Results  indicate  that  the  users  believe  the  Clipboard  is  Just  Right 
to  Too  Large.  A  significant  amount  of  respondents  indicated  they 
would  prefer  a  smaller  unit.  There  was  no  indication  that  the  unit 
is  too  small. 
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4.2  WEIGHT 
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4.2.1  Questionnaire  Evaluation  item: 

*  The  weight  was  appropriate  for  me  to  use  the  clipboard. 

4.2.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 

Percent  per  category  1  30%  I  7%  I  63%| 


Respondents  per  category  \  9  |  2  1 1 9 

Too  Just  Too 

Heavy  Right  Light 

4.2.3  Interpretation: 

*  Sixty  three  percent  of  respondents  indicated  that  the  weight  of  the 
unit  was  Just  Right  for  them.  Even  so,  a  significant  number  felt 
the  unit  was  too  heavy.  There  was  no  indication  that  the  unit  was 
too  light. 
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4.3  EDGE  SHAPE 
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4.3.1  Questionnaire  Evaluation  Item: 

*  The  shape  of  the  case  edges  was  appropriate  for  me  to  use  the 
Clipboard. 

4.3.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category  I 


Respondents  per  category 


Too 

Rounded 


100% 


1  29  I 

Just 

Right 


J_°J 

Too 

Angular 


4.3.3  Interpretation: 

'  All  respondents  indicated  that  the  shape  of  the  case  edges  was  just 
right. 
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4.4  SURFACE  TEXTURE 


4.4.1  Questionnaire  Evaluation  item: 

*  The  surface  texture  was  appropriate  for  me  to  use  the  Clipboard. 

4.4.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category 

52% 

48% 

Respondents  per  category 

15 

0 

14 

0 

0 

Too  Just  Too 

Smooth  Right  Rough 


4.4.3  Interpretation: 

'Results  indicate  that  52%  of  the  respondents  would  prefer  a  rougher 
texture,  such  as  a  rubber  case  or  more  knurling.  Respondents  felt 
that  the  unit  would  be  hard  to  hold  under  wet  conditions  or  when  the 
tank  was  on  the  move. 


4.5  LIGHT  SWITCH  LOCATION 


4.5.1  Questionnaire  Evaluation  Item: 

*  The  light  switch  was  in  the  appropriate  location  for  me  to  use  the 
Clipboard. 

4.5.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

97% 

3% 

Respondents  per  category 

0 

0 

29 

0 

1 

Too  Just  Too 


Far  Left  Right  Far  Right 


4.5.3  Interpretation: 

*  Results  show  that  97%  of  respondents  indicated  that  the  light 
switch  is  in  a  satisfactory  location  and  no  change  is  needed. 


4.6  LIGHT  SWITCH  PRESSURE 


4.6.1  Questionnaire  Evaluation  Item: 

*  The  pressure  of  the  light  switch  was  appropriate  for  me  to  use  the 
Clipboard. 

4.6.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

100% 

Respondents  per  category 

0 

0 

30 

MM 

MM 

Too  Just  Too 

Light  Right  Heavy 


4.6.3  Questionnaire  Interpretation: 

*  All  respondents  indicated  that  the  pressure  on  the  light  switch 
was  just  right. 


4.7  ON  SWTTCH  LOCATION 


4.7.1  Questionnaire  Evaluation  Item: 

'  The  ON  switch  was  in  the  appropriate  location  for  me  to  use  the 
Clipboard. 

4.7.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category 

100% 

Respondents  per  category 

0 

0 

29 

0 

0 

Too  Just  Too 

Far  Left  Right  Far  Right 


4.7.3  Interpretation: 

*  All  respondents  indicated  that  the  ON  switch  location  was  in  a  good 
position  and  presents  no  problems. 


4.8  ON  SWITCH  PRESSURE 


4.8.1  Questionnaire  Evaluation  Item: 

The  pressure  of  ON  switch  was  appropriate  for  me  to  use  the 
Clipboard. 

4.8.2  Questionnaire  Response: 

’  Total  number  of  respondents  for  this  item:  29 
*  Distribution  of  responses: 


Percent  per  category 

100% 

Respondents  per  category 

0 

0 

29 

0 

0 

Too  Just  Too 

Light  Right  Heavy 


4.8.3  Interpretation: 

*  Results  indicate  that  ali  respondents  had  no  trouble  with  the  ON 
switch  pressure  as  it  currently  is  on  the  Clipboard. 
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4.9  CASE  LABELS 


4.9.1  Questionnaire  Evaluation  item: 

'  The  labels  on  the  case  were  appropriate  for  me  to  use  the  Clipboard. 

4.9.2  Questionnaire  Response: 


*  Total  number  of  respondents  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category 

17% 

3% 

80% 

Respondents  per  category 

0 

0 

5 

1 

23 

Hard  to  Easy  to 

Understand  Understand 


4.9.3  Interpretation: 

*  Most  respondents  (80%)  indicated  they  tiad  no  problem 
understanding  the  Case  Labels.  Of  the  remaining  30%,  many 
indicated  that  the  labels  should  be  on  top  of  the  unit  or  of  different 
material.  The  majority  of  the  comments  from  the  30%  were  not 
applicable  to  the  evaluation  item  (hard  or  easy  to  understand)  but 
rather  appeared  to  represent  "personal  opinion"  regarding  label 
design. 
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4.10  BATTERY  CHARGER 

4.10.1  Questionnaire  Evaluation  item: 

*  The  Battery  Charger  was  appropriate  for  me  to  charge  the  Clipboard 

batteries.  ! 

i 

4.10.2  Questionnaire  Response: 

i 

j 

*  Total  number  of  respondents  for  this  item:  28 

*  Distribution  of  responses: 


Percent  per  category 

61% 

7% 

14% 

18% 

Respondents  per  category 

17 

2 

4 

0 

5 

Adequate  Not  Adequate 


4.10.3  Interpretation: 

'  The  results  suggest  that  most  respondents  (68%)  believed  the 
battery  chargers  were  adequate  for  their  needs.  However,  32% 
believed  the  charger  was  either  fair,  or  not  adequate  for  their 
needs.  The  majority  of  individuals  rating  the  chargers  other  than 
"Adequate"  indicated  they  would  like  a  charger  that  charges  faster. 
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Clipboard  Touch  Screen  Configuration  Questions 

4.11  TOUCH  AREA  SIZE 

4.11.1  Questionnaire  Evaluation  item: 

*  The  size  of  the  touch  areas  on  the  display  was  appropriate  for  me 
to  use  the  Clipboard. 

4.11.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

7% 

87% 

3% 

3% 

Respondents  per  category 

2 

0 

26 

1 

1 

Too 

Small 

Just 

Right 

Too 

Big 

4.11.3  Interpretation: 


*  A  significant  number  of  respondents  (87%)  indicated  that  the  size 
of  the  Touch  Areas  was  Just  Right.  There  were  no  consistent 
reasons  provided  by  the  remaining  13%  for  their  deviation  from 
Just  Right. 
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4.12  TOUCH  SENSITIVITY 


4.12.1  Questionnaire  Evaluation  Item: 

*  The  touch  screen  was  sensitive  enough  that  only  one  touch  was 
required  to  select  a  touch  area. 

4.12.2  Questionnaire  Response: 


*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category  37%  7% 

53% 

3% 

Respondents  per  category  1 1  2 

16 

0  1 

Too 

Just 

Not  Sensitive 

Sensitive 

i 

Right 

Enough 

4.12.3  Interpretation: 

*  The  majority  of  respondents  (53%)  believed  the  sensitivity  was  — 
Just  Right.  Another  44%  believed  the  touch  screen  was  Too 
Sensitive.  The  most  common  problems  were  when  they  accidently 
touched  the  screen,  or  when  moving  their  finger  to  a  touch  area, 
they  accidently  selected  another  area  Several  respondents 
suggested  touch  panels  such  as  those  on  many  new  microwaves. 


4.13  TOUCH  KEY  ARRANGEMENT 


4.13.1  Questionnaire  Evaluation  Item: 

*  The  arrangement  of  the  touch  keys  was  appropriate  for  me  to  use 
the  Clipboard. 

4.13.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

13% 

3% 

84% 

Respondents  per  category 

0 

0 

4 

1 

25 

Hard 

Easy 

To  Use  To  Use 

4.13.3  Interpretation: 

*  A  majority  of  respondents  (84%)  found  the  Touch  Key  Arrangement 
to  be  Easy  To  Use.  The  remaining  16%  indicated  that  the 
arrangement  was  between  Fair  to  Easy  To  Use.  A  few  respondents 
indicated  that  more  training  on  the  Clipboard  would  help,  m 
general,  most  felt  the  arrangement  was  adequate  and  logical. 
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4.14  TOUCH  SCREEN  MENUS  (UNDERSTANDABIUTY) 

4.14.1  Questionnaire  Evaluation  item: 


*  The  menus  presented  on  the  touch  screen  were  appropriate  for  me 
to  use  the  Clipboard  operating  modes  and  databases. 

4.14.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category 

90% 

3% 

7% 

Respondents  per  category 

26 

1 

2 

0 

0 

Easy  to  Hard  to 

Understand  Understand 


4.14.3  Interpretation: 

•  The  majority  of  respondents  (90%)  found  the  Menus  to  be  Easy  to 
Understand  when  using  the  Clipboard  operating  modes  and 
databases.  The  remaining  1 0%  indicated  a  need  for  more 
training. 


4.15  TOUCH  SCREEN  MENUS  (USEABIUTV) 

4.15.1  Questionnaire  Evaluation  Item: 


*  The  menus  presented  on  the  touch  screen  were  dear  and 
understandable,  and  appropriate  for  me  to  access  the  Clipboard 
operating  modes  and  databases. 

4.15.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

3% 

10% 

7% 

80% 

Respondents  per  category 

0 

1 

3 

2 

24 

Hard 

i 

Easy 

To  Use  To  Use 


4.15.3  Interpretation: 

*  The  majority  of  respondents  (80%)  found  the  Menus  Easy  and 
Useable.  Another  1 7%  found  the  Menus  Fair  to  Easy  and  3% 
experienced  trouble  using  the  menus.  Overall,  results  indicate  the 
Menus  are  Easy  To  Use. 


4.16  BRIGHTNESS 


4.16.1  Questionnaire  Evaluation  Item: 

*  The  display  brightness  was  appropriate  for  me  to  use  the  Clipboard. 

4.16.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category 

3% 

10% 

63% 

3% 

21% 

Respondents  per  category 

1 

3 

18 

1 

6 

Too  Just  Too 

Bright  Right  Dim 


4.16.3  Interpretation: 

*  Responses  for  this  item  varied  across  the  entire  scale.  Most 
respondents  (63%)  indicated  that  display  brightness  was  Just  Right. 
However,  37%  of  respondents  were  not  totally  pleased  with  the 
brightness.  The  majority  of  their  comments  focus  on  a  problem 
with  glare  from  the  sun.  At  certain  angles,  or  under  certain 
sunlight  conditions,  they  had  a  problem  reading  the  display. 


4.17  LETTER  SIZE 


4.17.1  Questionnaire  Evaluation  Item: 

0  The  size  of  the  letters  and  numbers  on  the  display  was  appropriate 
for  me  to  use  the  Clipboard. 

4.17.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

3% 

87% 

3% 

7% 

Respondents  per  category 

1 

26 

1 

2 

Too  Just  Too 

Small  Right  Big 


4.17.3  Interpretation: 

The  majority  (87%)  of  respondents  indicated  that  letter  size  was 
Just  Right.  Of  the  1 0%  that  felt  the  letters  were  Too  Big,  there  was 
an  indication  from  their  comments  that  they  wanted  more 
information  on  the  display,  thus,  smaller  letters  would  provide  the 
ability  to  display  more  information. 
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4.18  DISPLAY  LEGIBILITY  (DAYLIGHT) 

4.18.1  Questionnaire  Evaluation  Item: 

*  The  legibility  of  the  text  and  characters  shown  on  the  display  was 
appropriate  for  me  to  use  the  Clipboard  during  the  day. 

4.18.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category  13%|  13%  17%|  57% 


Respondents  per  category  4  |  4  |  5  0  17 

Not  Very 

Legible  Legible 


4.18.3  Interpretation: 

*  Results  vary  across  the  scale  for  this  item.  More  than  half  of  the 
respondents  had  no  problem  with  the  legibility  of  the  display.  A 
significant  number  of  respondents  indicated  that  display  legibility 
was  inadequate.  Some  attributed  it  to  glare  from  the  sun,  and 
others  indicated  a  true  need  for  more  letter/background  contrast. 


4.19  DISPLAY  LEGIBILITY  (NIGHTTIME) 

4.19.1  Questionnaire  Evaluation  item: 


*  The  legibility  of  the  text  and  characters  shown  on  the  display  was 
appropriate  for  me  to  use  the  Clipboard  at  night. 

4.19.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  22 

*  Distribution  of  responses: 


Percent  per  category 

9% 

14% 

77% 

Respondents  per  category 

2 

0  ( 

3 

0 

17 

Not  Very 

Legible  Legible 


4.19.3  Interpretation: 

*  Under  night-time  conditions  most  (77%)  believed  that  display 
legibility  was  Very  Legible.  Nine  percent  felt  it  was  Not  Legible. 


WiW'W-I  ■  AUiMiMil* 


420  ID  THE  EVALUATOR 

4.20.1  Questionnaire  Evaluation  Item: 

The  Clipboard  functions  were  appropriate  for  me  to  identify  the 
evaluator(s). 

4.20.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 
4  Distribution  of  responses: 


Percent  per  category 

3% 

3% 

94% 

Respondents  per  category 

0 

0 

1 

1 

28 

Difficult  Easy 

To  Use  To  Use 


4.20.3  Interpretation: 

Results  indicate  that  the  ID  The  Evaluator  function  is  adequate  and 
Easy  To  Use. 
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421  ID  THE  STUDENT 


4.21.1  Questionnaire  Evaluation  Item: 

*  The  Clipboard  functions  were  appropriate  for  me  to  identify  the 
student(s). 

4.21 .2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

4% 

6% 

90% 

Respondents  per  category 

1 

Ol 

2 

EH 

27 

Difficult  Easy 

To  Use  To  Use 


4.22  ID  THE  DRILL 


4.22.1  Questionnaire  Evaluation  Item: 

*  The  Clipboard  functions  were  appropriate  for  me  to  identify  the 
drill(s). 

4.22.2  Questionnaire  Response: 

*  Total  number  or  responses  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

3% 

97% 

0 

0 

1 

mm 

29 

Difficult  Easy 

To  Use  To  Use 


4.22.3  Interpretation: 

Results  indicate  that  the  ID  The  Drill  function  is  adequate  and  Easy 
To  Use. 
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4.23  TRAIN  ON  A  DRILL 


::'vJ 

.•:v, 

■■'VI 


‘-0 


bjS 


4.23.1  Questionnaire  Evaluation  Item: 

*  The  Clipboard  functions  were  appropriate  for  me  to  train  the 
drill(s)  in  the  field. 

4.23.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category  3% 

7% 

90% 

Respondents  per  category  1 

0 

2 

0 

26  | 

Difficult 
To  Use 


Easy 
To  Use 


•  .‘t 
cr.,h 

>?v’» 


SI 


4.23.3  Interpretation: 

*  Results  indicate  that  the  Train  On  A  Drill  function  is  perceived  to 
be  adequate  and  Easy  to  Use  in  a  field  environment. 
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4.24  EVALUATE  A  DRILL 

4.24.1  Questionnaire  Evaluation  Item: 

*  The  Clipboard  functions  were  appropriate  for  me  to  evaluate  the 
drill(s)  in  the  field. 

4.24.2  Questionnaire  Response: 

*  Total  number  of  responses  for  this  item:  28 

*  Distribution  of  responses: 


Percent  per  category  3%  3%  94% 


Respondents  per  category  M  |  0  I  0  |  1  I  26  1 

Difficult  Easy 

To  Use  To  Use 


4.24.3  Interpretation: 

*  Results  indicate  that  the  Evaluate  A  Drill  function  is  perceived  to 
be  adequate  and  Easy  to  Use  in  a  field  environment. 


4.25  REVIEW  A  PASS 


4.25.1  Questionnaire  Evaluation  Item: 

*  The  Clipboard  functions  were  appropriate  for  me  to  review  the 
student(s)  scores. 

4.25.2  Questionnaire  Response: 

'  Total  number  of  responses  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category 


Respondents  per  category 


7% 

93% 

0 

0 

2 

0 

27 

Difficult 
To  Use 


Easy 
To  Use 


4.25.3  Interpretation: 

*  Results  indicate  that  the  Review  A  Pass  function  is  adequate  and 
Easy  To  Use. 
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426  ENTRY  C0RRECT10N/CAN<^I1AT1^ 

4.26.1  Questionnaire  Evaluation  Item: 

*  The  Clipboard  functions  were  appropriate  for  me  to  correct  or 
cancel  any  entries  that  I  made. 

4.26.2  Questionnaire  Response: 

*  Total  number  of  responses  for  this  item:  30 
‘  Distribution  of  responses: 


Percent  per  category 

3% 

7% 

90% 

Respondents  per  category 

1 

0 

2 

0 

27 

Difficult  Easy 

To  Use  To  Use 


4.26.3  Interpretation: 

'  Results  indicate  that  the  Entry  Correction/Cancellation  function  is 
Easy  To  Use. 
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427  SCROLL  UP/DOWN 


4.27.1  Questionnaire  Evaluation  Item: 

*  The  Clipboard  functions  were  appropriate  for  me  to  scroll  up  and 
down  within  a  list. 

4.27.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

4% 

13% 

83% 

Respondents  per  category 

1 

wm 

KM 

WM 

25 

Difficult  Easy 

To  Use  To  Use 


4.27.3  Interpretation: 


1 


428  GQNO  GCVNOT  SEEN/NOT  DONE 

4.28.1  Questionnaire  Evaluation  item: 

*  The  Clipboard  functions  were  appropriate  for  me  to  identify  the 
tasks  that  were  GO,  NO  GO,  NOT  SEEN,  OR  NOT  DONE. 

4.28.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

3% 

3% 

94% 

Respondents  per  category 

0 

0 

1 

1 

28 

Difficult  Easy 

To  Use  To  Use 


4.28.3  Interpretation: 

*  Results  indicate  that  the  Go/No  Go/Not  Seen/Not  Done  function  was 
Easy  To  Use. 


429  STOPWATCH  (USEABIUTV) 

429.1  Questionnaire  Evaluation  Item: 

*  The  stopwatch  function  was  appropriate  for  me  to  time  events. 
4.29.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  29 
‘  Distribution  of  responses: 


Percent  per  category 


Respondents  per  category  |  0  j  0 

Difficult 
To  Use 


93% 


Easy 
To  Use 


v*i 

id 


<v 


4.29.3  Interpretation: 

The  results  indicate  that  the  Stopwatch  function  is  Easy  To  Use. 
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4.30  STOPWATCH  (ADEQUACY) 

4.30.1  Questionnaire  Evaluation  item: 


*  The  Stopwatch  function  was  appropriate  for  me  to  time  events. 

4.30.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  28 

*  Distribution  of  responses: 


Percent  per  category 

82% 

4% 

14% 

Respondents  per  category 

23 

0 

1 

0 

4 

Adequate  Not  Adequate 


4.30.3  Interpretation: 

Results  indicate  that  the  Stopwatch  function  was  adequate  for  its 
use.  However,  some  respondents  expressed  the  desire  for  the 
Stopwatch  function  to  include  the  ability  to  enter  a  specific  time 
and  have  the  unit  count-down  that  time. 
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4.31  OPERATING  TIME  REMAINING 


.  •.* 


| 

I 


4.31 .1  Questionnaire  Evaluation  Item: 

*  The  Clipboard  functions  were  appropriate  for  me  to  determine  the 
operating  time  remaining. 

4.31 .2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category 

17% 

83% 

Respondents  per  category 

5 

0 

0 

0 

24 

Difficult  Easy 

To  Use  To  Use 


5  4.31.3  Interpretation: 

i 

£  9  Overall,  results  indicate  that  the  Operating  Time  Remaining 

I  function  is  Easy  To  Use.  Comments  from  those  who  stated  they  had 

?  difficulty  using  this  function  concerned  such  issues  as  a  need  for 

J  longer  battery  life  or  the  ease  with  which  one  could  "fool"  the  Time 

Remaining  display,  rather  than  difficulty  using  the  Operating  Time 
j  Remaining  function. 


4.32  DATE/TIME 


4.32.1  Questionnaire  Evaluation  Item: 

*  The  Clipboard  functions  were  appropriate  for  me  to  use  the 
calendar  and  clock. 

4.32.2  Questionnaire  Response: 

*  Total  number  of  responses  for  this  item:  30 

*  Distribution  of  responses: 


Percent  per  category 

7% 

93% 

Respondents  per  category 

0 

0 

2 

0 

28 

Difficult  Easy 

To  Use  To  Use 


4.32.3  Interpretation: 

*  Results  indicate  that  the  Date/Time  function  was  Easy  To  Use. 


I'*  i«-«  »■ 


5.33  AUTOMATIC  OFF 

5.33.1  Questionnaire  Evaluation  item: 

*  The  display  turns  off  automatically  after  3  minutes  if  no  key  is 
pressed.  This  function  was  adequate  for  me  to  use  the  Clipboard. 

4.33.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category 

7% 

3% 

90% 

Respondents  per  category 

0 

0 

2 

1 

26 

Interfered  Did  Not  Interfere 

with  my  job  with  my  job 


4.33.3  Interpretation: 

•  Results  indicate  that  the  Automatic  Off  function  did  not  interfere 
with  the  respondent's  job.  There  was  a  10%  deviation  from  that 
statement  but  no  consistent  reasons  were  provided. 
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Questionnaire  Evaluation  Item:  Please  rate  the  overall  useability  of  the 

Clipboard  under  the  following  conditions: 


4.34  DAYTIME 

4.34.1  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  29 

*  Distribution  of  responses: 


Percent  per  category 

52% 

21% 

17% 

7% 

3% 

Respondents  per  category 

15 

6 

5 

2 

1 

Very  Easy  Very  Difficult 

to  Use  to  Use 


4.34.2  Interpretation: 

*  Results  indicate  that  most  respondents  (52%)  believed  the 
Clipboard  was  Very  Easy  to  Use  under  daylight  conditions.  Results 
from  the  remaining  48%  spread  across  the  scale.  Comments  from 
the  respondents  indicate  that  glare  from  the  sun  caused  the  display 
to  be  difficult  to  read  at  times. 
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4.35  NIGHTTIME 


4.35.1  Questionnaire  Response: 

#  Total  number  of  respondents  for  this  item:  25 

*  Distribution  of  responses: 


Percent  per  category 

48% 

32% 

4% 

4% 

12% 

Respondents  per  category 

12 

8 

1 

1 

3 

Very  Easy  Very  Difficult 

to  Use  to  Use 


4.35.2  Interpretation: 

*  Most  respondents  (80%)  found  the  Clipboard  Easy  to  Very  Easy  to 
Use  under  nighttime  conditions.  Some  respondents  indicated  that 
the  unit  was  too  bright  at  night  and  that  the  user  should  have  the 
ability  to  turn  the  light  OFF  and  ON. 


4.36  EVALUATING  MOBILE  FORCES 


4.36.1  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  24 
'  Distribution  of  responses: 


Percent  per  category 

58% 

4% 

13% 

4% 

21% 

Respondents  per  category 

14 

1 

3 

1 

5 

Very  Easy  Very  Difficult 

to  Use  to  Use 


4.36.2  Interpretation: 

*  The  majority  of  respondents  indicated  that  they  perceived  the 
Clipboard  would  be  Very  Easy  to  Use  while  evaluating  mobile  force. 
The  remaining  Results  vary  across  the  remainder  of  the  scale. 
Comments  from  some  respondents  indicate  that  they  felt  the 
Clipboard  might  be  difficult  to  use  while  in  a  vehicle  in  motion. 
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4.37  EVALUATING  STATIONARY  f=ORCES 
4.37.1  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  26 

*  Distribution  of  responses: 

Percent  per  category  85%  12% 
Respondents  per  category  22  3  0 

Very  Easy 
to  Use 


Very  Difficult 
to  Use 


4.37.2  Interpretation: 

*  Results  indicate  that  the  Clipboard  is  perceived  to  be  Easy  to  Use 
for  evaluating  stationary  forces. 
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4.38  EVALUATING  DISCRETE  EVENTS 

tS* 

4.38.1  Questionnaire  Response: 

«!* 

♦ 

*>?* 

*  Total  number  of  respondents  for  this  item:  27 

• 

'  V* 

*  Distribution  of  responses: 

'V. 

,V. 

Percent  per  category  1 74%  1 15%  |4%  | 

|  7%  | 

>4 

a 


m 


% 

$ 


Respondents  per  category  | 20  4 

Very  Easy 
to  Use 


Very  Difficult 
to  Use 


4.38.2  Interpretation: 

*  Results  indicate  that  the  Clipboard  is  Easy  to  use  while  evaluating 
discrete  events. 


4.39  EVALUATING  EVENTS  IN  A  SERIES  OF  OTHER  EVENTS 
4.39.1  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  25 

*  Distribution  of  responses: 


Percent  per  category 

76% 

12% 

8% 

4% 

Respondents  per  category 

19 

3 

2 

0 

1 

Very  Easy  Very  Difficult 

to  Use  to  Use 


4.39.2  Interpretation: 


*  Results  indicate  that  most  respondents  (76%)  believed  that  the 
Clipboard  was  Very  Easy  to  Use. 


4.40  FIELD  ENVIRONMENT 


4.40.1  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  18 

*  Distribution  of  responses: 


Percent  per  category 

28% 

17% 

17% 

33% 

5% 

Respondents  per  category 

5 

3 

3 

6 

1 

Very  Easy  Very  Difficult 

to  Use  to  Use 


4.40.2  Interpretation: 

*  Results  indicate  that  there  are  no  consistent  trends  in  user's 
perceptions  of  the  difficulty  of  using  the  Clipboard  in  a  field 
environment.  What  the  comments  and  results  do  suggest  is  that 
some  respondents  believe  that  the  Clipboard  would  be  difficult  to 
use  in  the  field.  Many  suggested  that  either  a  handle  or  some  type 
of  grip  would  be  useful.  Others  felt  the  unit  was  not  sufficiently 
ruggedized,  and  that  the  casing  should  be  made  of  a  different 
material  (e.g.,  rubber  or  aluminum).  Still  others  indicated  that  for 
use  in  the  field  environment  the  openings  around  the  plug,  ON 
switch,  and  LIGHT  switch  would  need  to  be  sealed. 


4.41  SET  ECS  INITIAL  CONDITIONS  (ENTER  EVALUATOR  NAMES) 

4.41.1  Questionnaire  Evaluation  Item: 

*  The  Training  Base  Station  functions  were  appropriate  for  me  to  set 
the  initial  conditions  -  Enter  Evaluator  Names. 

4.41 .2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  22 

*  Distribution  of  responses: 


Percent  per  category 

4% 

4% 

92% 

Respondents  per  category 

0 

0 

1 

1 

20 

Difficult 
to  Use 


Easy  to 
Use 


4.41.3  Interpretation: 

*  Results  indicate  that  the  Enter  Evaluator  Names  function  was  Easy 
to  Use. 


4.42  SET  ECS  INITIAL  CONDITIONS  (ENTER  SOLDIER  NAMES) 

4.42.1  Questionnaire  Evaluation  Item: 


'  The  Training  Base  Station  functions  were  appropriate  for  me  to  set 
the  initial  conditions  *  Enter  Soldier  Names. 


4.42.2  Questionnaire  Response: 

*  Total  number  of  responses  for  this  item:  23 

*  Distribution  of  responses: 


Percent  per  category 

9% 

4% 

87% 

Respondents  per  category 

0 

0 

2 

1 

20 

Difficult  Easy  to 

to  Use  Use 


4.42.3  Interpretation: 

Results  indicate  that  the  Enter  Soldier  Names  function  is  Easy  to 
Use. 


4.43  SELECT  ECS  INITIAL  CONDmONS  (SELECT  TRAINING  GUIDE) 

4.43.1  Questionnaire  Evaluation  Item: 


The  Training  Base  Station  functions  were  appropriate  for  me  to  set 
the  initial  conditions  -  Select  Training  Guide. 

4.43.2  Questionnaire  Response: 

Total  number  of  respondents  for  this  item:  24 
*  Distribution  of  responses: 


Percent  per  category 

4% 

4% 

92% 

Respondents  per  category 

0 

0 

1 

1 

22 

Difficult  Easy  to 

to  Use  Use 


4.43.3  Interpretation: 

Results  indicate  that  the  Select  Training  Guide  function  is  Easy  to 
Use. 


4.44  DIAGNOSE  ECS  STATUS 


4.44.1  Questionnaire  Evaluation  Item: 

*  The  Training  Base  Station  functions  were  appropriate  for  me  to 
diagnose  the  ECS  operational  status. 

4.44.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  23 

*  Distribution  of  responses: 


Percent  per  category 

4% 

96% 

Respondents  per  category 

0 

0 

1 

0 

22 

Difficult  Easy  to 


to  Use 


Use 


4.44.3  Interpretation: 


*  Results  indicate  that  the  Diagnose  ECS  Status  function  is  Easy  to 
Use. 


47 


445  TRANSFER  ECS  CONDmONS  TO  CUPBOARD 

4.45.1  Questionnaire  Evaluation  item: 

*  The  Training  Base  Station  functions  were  appropriate  for  me  to 
transfer  the  initial  conditions  to  the  Electronic  Clipboard. 

4.45.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  24 

*  Distribution  of  responses: 


Percent  per  category  4%  96% 


Respondents  per  category  1 0  0  0  1  23 

Difficult  Easy  to 

to  Use  Use 


4.45.3  Interpretation: 

*  Results  indicate  that  the  Transfer  ECS  Conditions  To  Clipboard 
function  is  Easy  to  Use. 


or. 


4.46  UPLOAD  TRAINING  DATA 

4.46.1  Questionnaire  Evaluation  Item: 

The  Training  Base  Station  functions  were  appropriate  for  me  to 
upload  scores  from  the  Clipboard. 

4.46.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  22 

*  Distribution  of  responses: 


Percent  per  category 

100% 

Respondents  per  category 

0 

0 

0 

0 

22 

Difficult 
to  Use 


Easy  to 
Use 


4.46.3  Interpretation: 

Results  indicate  that  the  Upload  Training  Data  function  is  Easy  to 
Use. 


4.47  DISPLAY  OF  SUMMARY  REPORTS 


4.47.1  Questionnaire  Evaluation  Item: 

*  The  displayed  presentation  of  summary  score  reports  was 
appropriate  for  me  to  do  my  job. 

4.47.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  18 

*  Distribution  of  responses: 


Percent  per  category 

100% 

Respondents  per  category 

0 

0 

0 

0 

18 

Difficult  Easy  to 

to  Use  Use 


4.47.3  Interpretation: 

*  Results  indicate  that  the  Display  of  Summary  Report  is  Easy  to  Use. 
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4.48  El^mY(X)RRECT10N £AN(X1±ATON 

4.48.1  Questionnaire  Evaluation  Item: 

*  The  Training  Base  Station  functions  were  appropriate  for  me  to 
correct  or  cancel  any  entries  that  I  made. 

4.48.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  22 

*  Distribution  of  responses: 


Percent  per  category 

4% 

4% 

92% 

Respondents  per  category 

1 

1 

20 

0 

0 

Too  Just  Very 

Complicated  Right  Unclear 


4.48.3  Interpretation: 

Results  indicate  that  the  Entry  Correction/Cancellation  functions 
was  Easy  to  Use. 


4.49  ECS  -  TBS  CONNECTION 


\ 


4.49.1  Questionnaire  Evaluation  item: 

*  The  connection  between  the  Electronic  Clipboard  and  the  Training 
Base  Station  was  appropriate  for  me  to  successfully  connect  the 
two  devices. 

4.49.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  24 

*  Distribution  of  responses: 


Percent  per  category 

4% 

92% 

4% 

Respondents  per  category 

0 

1 

22 

0 

1 

Too  Just  Very 

Complicated  Right  Unclear 


4.49.3  Interpretation: 

*  Results  indicate  that  the  ECS  -  TBS  Connection  is  Easy  to  Use. 


450  TBS  MENU  OPTIONS  (USEABILTTY) 

4.50.1  Questionnaire  Evaluation  Item: 

*  The  Training  Base  Station  menu  options  were  appropriate  for  me  to 
do  my  job. 

4.50.2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  23 

*  Distribution  of  responses: 


Percent  per  category 


Respondents  per  category 


Difficult 
to  Use 


9%  9%  4%  78% 


-  , _ L2U 

Easy  to 
Use 


4.50.3  Interpretation: 

*  Most  respondents  (78%)  believed  the  TBS  Menu  Options  were  Easy  to 
Use.  Some  of  those  that  had  difficulty,  indicated  more  training 
would  be  helpful. 
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4.51  TBS  MENU  OPTIONS  (ADEQUACY) 

4.51.1  Questionnaire  Evaluation  item: 


*  The  Training  Base  Station  menu  options  were  appropriate  for  me  to 
select  the  options  I  wanted. 

4.51 .2  Questionnaire  Response: 

*  Total  number  of  respondents  for  this  item:  21 

*  Distribution  of  responses: 


Percent  per  category 

81% 

5% 

9% 

5% 

Respondents  per  category 

17 

1 

2 

0 

1 

Adequate  Not  Adequate 


4.51.3  Interpretation: 

*  Most  respondents  indicated  that  the  Menu  Options  were  Adequate 
for  their  needs.  The  individual  that  found  the  options  somewhat 
inadequate  provided  no  comments  to  clarify  his  rating. 
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5.0  SUMMARY  OF  BNCOC  EVALUATION 

Due  to  the  tight  time  schedules  of  the  BNCOC  evaluators,  it  was  found 
more  expedient  to  extract  evaluation  data  from  them  in  a  more  tree-form 
format,  by  the  use  of  verbal  questions  (modeled  after  questions  on  the 
questionnaire)  and  "role  playing",  whereby  the  evaluators  ran  through 
scenarios  for  using  the  Clipboard  under  operational  conditions  and 
reported  their  thoughts  and  impressions  as  to  the  strengths  and 
weaknesses  of  the  current  design.  There  were  also  BNCOC  evaluators  who 
used  the  Clipboard  during  Gunnery  and  Land  Navigation  exercises,  and  under 
day  and  night  conditions.  BNCOC  evaluator  recommendations  are 
summarized  below: 

o  Provide  faster  scroll 

o  Use  non-glare  surface  on  display 

o  Provide  positive  feedback  on  keyboard  (so  user  can  tell  when  a  key 
has  been  pressed 

o  Develop  more  ruggedized  package 
o  Include  RF  shielding 

o  Make  sure  all  switches  and  connectors  are  sealed  (i.e., 
against  dirt  and  moisture) 

o  Make  case  scratch  resistant 

o  Make  case  fingerprint  resistant 

o  Make  unit  more  resistant  to  damage  due  to  environmental  and 
handling  factors  (e.g.,  temperature,  humidity,  drop) 

o  Increase  brightness  of  display  (for  daytime  use) 

o  Provide  capability  to  charge  unit  off  of  DC  power  supply 


o  Add  a  numeric  keypad 


o  Link  Train  on  Drill  and  Evaluate  a  Drill  modes  in  such  a  way  that 
user  can  go  rapidly  back  and  forth  between  them  without  losing  his 


o  Do  not  automatically  route  user  into  next  unscored  pass  each  time 
Train  On  Drill  mode  is  requested.  Make  soldier  indicate  that  he  has 
completed  a  pass  before  he  can  access  the  next  pass. 

o  Consider  making  time  for  backlight  timeout  selectable 

o  Consider  adding  a  software  dimmer  switch  for  the  backlight 

o  Improve  the  means  of  indicating  when  a  given  key  selection  has 
been  made  (e.g.,  use  flashing  of  entire  key  selected,  or  inverse 
video) 

o  Find  a  better  name  for  the  function  "Get  More  Words" 

o  Make  error  messages  more  obvious 

o  Show  student  name  and  pass  number  at  all  times  in  the  Evaluate  a 
Drill  mode 

o  Add  an  "OFF"  switch  capability  for  the  backlight  (e.g.,  toggle  the 
Light  switch) 

o  Provide  an  "intelligent"  battery  charger  so  there  will  be  no  worry 
about  overcharging 

o  Provide  a  battery  charger  capable  of  providing  a  full  charge  to  the 
Clipboard  in  under  eight  hours 

o  Make  the  scroll  arrows  more  visible 

o  Add  a  "clock  option"  to  the  TBS  so  that  the  date  and  time  passed  to 
the  Clipboard  will  be  accurate  and  consistent  across  users,  and  not 
dependent  upon  the  entry  of  date  and  time  by  each  individual  user 
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o  Consider  adding  a  "set  marker"  function  that  will  allow  users  to 
define  their  own  places  in  the  database  to  which  they  can  rapidly 
jump 

o  Consider  making  the  unit  smaller  and  lighter 

o  Provide  a  cap  for  the  RS-232  connector 

o  Make  the  unit  easier  to  hold  on  to  (e.g.,  add  handle  or  holster; 
change  surface  texture  to  make  unit  easier  to  hold  on  to) 

o  Make  it  possible  to  change  batteries  in  the  field 

o  Increase  Clipboard  memory  retention  time  (so  will  retain  scores 
more  than  five  days) 

o  Increase  Clipboard  memory  so  Clipboard  can  store  multiple  training 
guides 


6.0  CONCLUSIONS  AND  RECOMMENDATIONS 


It  is  not  the  purpose  of  this  section  to  recommend  a  design  for  a  new 
improved  version  of  the  Clipboard  and  the  Training  Base  Station,  but  rather 
to  recommend  features  to  be  considered  in  anv  new  design  effort  based 
upon  knowledge  gained  during  the  Fort  Knox  TIMS  field  evaluation.  Such 
features  are  presented  below  in  two  categories: 

o  Those  considered  to  be  very  important  to  include  in  a  new  design 

o  Those  considered  important  for  future  consideration  but  optional 
for  the  next  generation  design  (i.e.,  not  absolutely  essential) 

It  is  important  to  realize  that  some  features  may  interact  or  be  mutually 
exclusive  and  such  issues  must  be  addressed  during  any  future  design 
efforts. 

6.1  Essential  Features  to  Include  In  Future  Designs 

o  High  contrast  display 

o  Positive  feedback  on  touch  panel  (so  user  can  tell  when  keys  have 
been  pressed) 

o  Increased  ruggedization 

o  Faster  scroll 

o  Link  Train  on  Drill  and  Evaluate  a  Drill  modes  in  such  a  way  that 
user  can  go  rapidly  back  and  forth  between  them  without  losing  his 
place 

o  Do  not  automatically  route  user  into  next  unscored  pass  each  time 
Train  On  Drill  mode  is  requested.  Make  soldier  indicate  that  he  has 
completed  a  pass  before  he  can  access  the  next  pass. 

o  Improve  the  means  of  indicating  when  a  given  key  selection  has 
been  made  (e.g.,  use  flashing  of  entire  key  selected,  or  inverse 
video) 


o  Show  student  name  and  pass  number  at  all  times  in  the  Evaluate  a 
Drill  mode 


o  Add  an  "OFF"  switch  capability  for  the  backlight  (e.g.,  toggle  the 
Light  switch) 


o  Increase  Clipboard  memory  retention  time  from  five  days  to  over 
one  month 


o  Provide  intensity  control  for  electroluminescent  backlight 
o  Make  batteries  field  replaceable 

o  Increase  Clipboard  memory  so  Clipboard  can  store  multiple  training 
guides  simultaneously 

o  Provide  a  capability  to  charge  Clipboard  from  a  DC  power  source 

o  Add  a  numeric  keyboard  (software  keys) 

o  Improve  the  means  of  indicating  when  a  given  key  selection  has 
been  made  (e.g.,  use  flashing  of  entire  key  selected,  or  inverse 
vi^ao) 

o  Make  displayed  error  messages  more  obvious 

o  Add  a  "set  marker"  function  that  will  allow  users  to 
define  their  own  places  in  the  database  to  which  they  can  rapidly 


In  addition  to  the  above,  the  following  recommendations  for  improvements 
are  made  by  the  developers  (Perceptronics): 

o  Consider  adding  a  capability  to  download  not  only  databases  but 
also  applications  programs  to  the  Clipboard.  This  will  allow  the 
Clipboard  to  be  easily  configured  to  support  multiple  requirements. 

o  Consider  rehosting  the  Training  Base  Station  onto  other  computers 
currently  in  use  by  major  user  communities; 


APPENDIX  A 


SAMPLE  QUESTIONNAIRE 
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Training  Information  Management  System 

(TIMS) 

Evaluation  Questionnaire 

TIMS  is  designed  to  be  part  of  a  larger  Army  training  management  system,  in  which 
data  (e.g.,  student  scores,  training  time)  gathered  during  a  training  course  can  be 
used  not  only  for  evaluation  of  the  individual  students  in  the  course,  but  also  as 
feedback  to  the  course  developers  for  improving  the  course  itself.  The  following 
diagram  depicts  the  envisioned  Army  training  management  system  in  which  the 
TIMS,  with  its  Electronic  Clipboard  and  Training  Base  Station  components, 
provides  a  "field  data  collection  capability".  The  TIMS  is  the  first  part  of  the  larger 
training  management  system  to  be  developed. 


The  purpose  of  this  Questionnaire  is  to  determine  how  well  the  TIMS  (including  the 
Electronic  Clipboard  and  the  Training  Base  Station)  works  for  collecting,  storing, 
and  summarizing  training  data  during  a  training  course.  Your  responses  to  this 
questionnaire  will  help  the  developers  of  TIMS  to  improve  the  system. 

Your  responses  will  be  kept  confidential,  and  no  records  will  be  kept  that  can  be 
associated  with  you  or  your  personnel  records.  When  you  answer  the  "Biographical 
Questionnaire,"  you  need  not  include  any  information  that  identifies  you  personally. 


A  series  of  statements  regarding  various  aspects  of  the  TIMS  are  presented  on  the 
following  pages.  A  TIMS  evaluator  will  discuss  each  of  the  statements  with  you,  and 
ask  you  to  place  an  "X"  along  a  scale  indicating  your  degree  of  agreement  or 
disagreement  with  each  statement  The  TIMS  evaluator  may  also  ask  you  questions 
designed  to  help  you  express  your  opinions  in  greater  detail. 


Clipboard  Physical  Configuration 

Size  -  The  overall  size  (lengtl 


The  overall  size  (length,  width,  and  thickness)  was 
appropriate  for  me  to  use  the  Clipboard. 


J _ I. 

Just  Right 


Too  Big  Just  Right  Too  Small 

If  the  Size  was  not  Just  Right,  please  explain  how  it  should  be  different: 


Weight  - 


The  weight  was  appropriate  for  me  to  use  the  Clipboard. 


Too  Heavy  Just  Right  Too  Light 

If  the  Weight  was  not  Just  Right,  please  explain  how  it  should  be  different: 


C«wW 

»3 

'I'W 


Clipboard  Physical  Configuration  (continued) 

Edge  Shape  -  The  shape  of  the  case  edges  was  appropriate  for  me  to  use  the 

Clipboard. 

I _ I _ I _ I _ I _ I 

Too  Rounded  Just  Right  Too  Angular 

If  the  Shape  of  the  Edges  was  not  Just  Right,  please  explain  how  it  should  be 
different* 


Surface  Texture  - 


The  surface  texture  was  appropriate  for  me  to  use  the 
Clipboard. 


I _ I _ I _ I _ I _ I 

Too  Smooth  Just  Right  Too  Rough 

If  the  Surface  Texture  was  not  Just  Right,  please  explain  how  it  should  be 
different: 


Clipboard  Physical  Configuration  (continued) 

Light  Switch  location-  The  light  switch  was  in  the  appropriate  location  for  me 

to  use  the  Clipboard. 

I _ I _ I _ I _ I _ I 

Too  Far  Left  Just  Right  Too  Far  Right 

If  the  Light  Switch  location  was  not  Just  Right,  please  explain  how  it  should 
be  different: 


Light  Switch  pressure  -  The  pressure  of  light  switch  was  appropriate  for  me  to 

use  die  Clipboard. 


Too  Light  Just  Right  Too  Heavy 

If  the  Light  Switch  pressure  was  not  Just  Right,  please  explain  how  it  should 
be  different: 
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Clipboard  Physical  Configuration  (continued) 

ON  Switch  location-  The  ON  switch  was  in  the  appropriate  location  for  me  to 

use  the  Clipboard. 

I _ I _ I _ I _ I _ I 

Too  Far  Left  Just  Right  Too  Far  Right 


If  the  ON  Switch  location  was  not  Just  Right,  please  explain  how  it  should  be 
different 


ON  Switch  pressure  -  The  pressure  of  ON  switch  was  appropriate  for  me  to 

use  the  Clipboard. 

I _ I _ I _ I _ L. _ I 

Too  Light  Just  Right  Too  Heavy 

If  the  ON  Switch  pressure  was  not  Just  Right,  please  explain  how  it  should  be 
different 


Clipboard  Physical  Configuration  (continued) 


Case  Labels  - 


The  labels  on  the  case  were  appropriate  for  me  to  use  the 
Clipboard. 


Haiti  to  Understand 


Easy  to  Understand 


If  the  Case  Labels  were  not  Easy  to  Understand,  please  explain  how  they 
should  be  different: 


Battery  Charger  -  The  Battery  Charger  was  appropriate  for  me  to  charge  the 

Clipboard  batteries. 


Adequate 


Not  Adequate 


If  the  Battery  Charger  was  not  Adequate,  please  explain  how  it  should  be 
different: 


Clipboard  Touch  Screen  Configuration 


Touch  Area  Size  -  The  size  of  the  touch  areas  on  the  display  was  appropriate 

for  me  to  use  the  Clipboard. 


Too  Small  Just  Right  Too  Big 


If  the  Size  of  the  Touch  Areas  was  not  Just  Right,  please  explain  how  it 
should  be  different: 


Touch  Sensitivity  -  The  touch  screen  was  sensitive  enough  that  only  one  touch 

was  required  to  select  a  touch  area. 

I _ I _ I _ I _ I _ I 

Too  Sensitive  Just  Right  Not  Sensitive  Enough 

If  the  Touch  Sensitivity  was  not  Just  Right,  please  explain  how  it  should  be 
different: 


Clipboard  Touch  Screen  Configuration  (continued) 


Touch  Key  Arrangement  -  The  arrangement  of  the  touch  keys  was  appropriate 

for  me  to  use  the  Clipboard. 
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Clipboard  Touch  Screen  Configuration  (continued) 


Touch  Screen  Menus  - 
(Useability) 


Hard  to  Use 


The  menus  presented  on  the  touch  screen  were  clear 
and  understandable,  and  appropriate  for  me  to  access 
the  Clipboard  operating  modes  and  databases. 


Easy  to  Use 


If  the  wording  or  organization  of  the  Touch  Screen  Menus  did  not  make  them 
Easy  to  Use,  please  explain  how  it  should  be  different* 


Clipboard  Display 

Brightness  -  The  display  brightness  was  appropriate  for  me  to  use  the 

Clipboard. 


Too  Bright  Just  Right  Too  Dim 

If  the  Display  Brightness  was  not  Just  Right,  please  explain  how  it  should  be 
different: 


! 

i 

!  Letter  Size  -  The  size  of  the  letters  and  numbers  on  the  display  was 

I  appropriate  for  me  to  use  the  Clipboard. 


Too  Small  Just  Right  Too  Big 


If  the  Letter  Size  was  not  Just  Right,  please  explain  how  it  should  be  different: 


Clipboard  Display  (continued) 


Display  Legibility  -  The  legibility  of  the  text  and  characters  shown  on  the 
(Daylight)  display  was  appropriate  for  me  to  use  the  Clipboard  during 

the  day. 


Not  Legible  Very  Legible 

If  the  text  and  characters  shown  on  the  screen  were  not  Very  Legible 
during  the  day,  please  explain  the  problem(s)  observed: 


i 


Display  Legibility  -  The  legibility  of  the  text  and  characters  shown  on  the 
(Nighttime)  display  was  appropriate  for  me  to  use  the  Clipboard  at 

night 


Not  Legible  Very  Legible 

If  the  ext  and  characters  shown  on  the  screen  were  not  Very  Legible  at 
night,  please  explain  the  problem(s)  observed: 


II 
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Clipboard  Functions 

ID  the  Evaluator  -  The  Clipboard  functions  were  appropriate  for  me  to  identify 

the  evaluators). 


Difficult  to  Use 


Easy  to  Use 


If  the  ID  the  Evaluator  Function  was  not  Easy  to  Use,  please  explain  how  it 
should  be  different: 


ID  the  Student 


Difficult  to  Use 


The  Clipboard  functions  were  appropriate  for  me  to  identify 
the  student(s). 


Easy  to  Use 


If  the  ID  the  Student  Function  was  not  Easy  to  Use,  please  explain  how  it 
should  be  different* 


a 


a 


Clipboard  Functions  (continued) 


ID  the  Drill  - 


Difficult  to  Use 


The  Clipboard  functions  were  appropriate  for  me  to  identify 
the  drill(s). 


Easy  to  Use 


If  the  ID  the  Drill  Function  was  not  Easy  to  Use,  please  explain  how  it  should 
be  different: 


Train  on  a  Drill  -  The  Clipboard  functions  were  appropriate  for  me  to  train 

the  drill(s)  in  the  field. 


Difficult  to  Use 


Easy  to  Use 


If  the  Train  on  a  Drill  Function  was  not  Easy  to  Use,  please  explain  how  it 
should  be  different: 


mm 


Clipboard  Functions  (continued) 

Evaluate  a  Drill  -  The  Clipboard  functions  were  appropriate  for  me  to 

evaluate  the  drill(s)  in  the  field. 


Difficult  to  Use  Easy  to  Use 

If  the  Evaluate  a  Drill  Function  was  not  Easy  to  Use,  please  explain  how  it 
should  be  different: 


Review  a  Pass  -  The  Clipboard  functions  were  appropriate  for  me  to  review 

the  students)  scores. 

I _ I _ I _ I _ I _ I 

Difficult  to  Use  Easy  to  Use 

If  the  Review  a  Pass  Function  was  not  Easy  to  Use,  please  explain  how  it 
should  be  different: 


Clipboard  Functions  (continued) 

Entry  Correction/  -  The  Clipboard  functions  were  appropriate  for  me  to  correct 
Cancellation  or  cancel  any  entries  that  I  made. 

I _ I _ I _ I _ I _ I 

Difficult  to  Use  Easy  to  Use 

If  the  Entry  Correction/Cancellation  was  not  Easy  to  Use,  please  explain 
how  it  should  be  different: 


Scroll  Up/Down  -  The  Clipboard  functions  were  appropriate  for  me  to  scroll 

up  and  down  within  a  list 


Difficult  to  Use 


Easy  to  Use 


If  the  Scroll  Up/Down  was  not  Easy  to  Use,  please  explain  how  it  should  be 
different 


Clipboard  Functions  (continued) 


Go/No  Go  *  The  Clipboard  functions  were  appropriate  for  me  to  identify 

Not  Seen/Not  Done  the  tasks  that  were  GO,  NO  GO,  NOT  SEEN,  or  NOT 

DONE. 


Difficult  to  Use  Easy  to  Use 

If  the  Go,  No/Go,  Not  Seen/Not  Done  functions  were  not  Easy  to  Use, 
please  explain  how  they  should  be  different: 


Stopwatch  -  The  Stopwatch  function  was  appropriate  for  me  to  time 

(Usability)  events. 


Difficult  to  Use  Easy  to  Use 

If  the  Stopwatch  was  not  Easy  to  Use,  please  explain  how  it  should  be  different: 


Stopwatch  - 
(Adequacy) 

I _ I _ I _ I _ I _ 

Adequate  Not  Adequate 


If  the  Stopwatch  was  not  Adequate  for  timing  events,  please  explain  how  it 
should  be  different: 


Operating  Time  -  The  Clipboard  functions  were  appropriate  for  me  to 
Remaining  determine  the  operating  time  remaining. 

I _ I _ I _ I _ I _ 

Difficult  to  Use  Easy  to  Use 


The  Stopwatch  function  was  appropriate  for  me  to  time 
events. 


If  the  Operating  Time  Remaining  was  not  Easy  to  Use,  please  explain  how  it 
should  be  different: 


Clipboard  Functions  (continued) 

Date/Time  -  The  Clipboard  functions  were  appropriate  for  me  to  use  the 

calendar  and  clock. 


Difficult  to  Use  Easy  to  Use 


If  the  Date/Time  was  not  Easy  to  Use,  please  explain  how  it  should  be 
different* 


Automatic  OFF  -  The  display  turns  off  automatically  after  3  minutes  if  no  key 

is  pressed.  This  function  was  adequate  for  me  to  use  the 
Clipboard . 


I _ I _ I _ I _ I _ I 

Interfeired  with  Did  not  interfere 

my  job  with  my  job 

If  the  Automatic  OFF  interfered  with  your  job,  please  explain  how  it  should  be 
different 


Overall  Useability  Please  rate  the  overall  useability  of  the  Clipboard 

under  the  following  conditions. 

Rate  each  question,  using  numbers  from  0  to  5,  as 
follows: 


1 

2 

3 

4  5 

Very  Easy  to  Use 

Very  Difficult 

to  Use 

Condition 

Rating 

Comments 

Daytime 

Nighttime 

Evaluating  Mobile  Forces 

Evaluating  Stationary  Forces 

Evaluating  Discrete  Events 

Evaluating  Events  in  a  Series 
of  other  Events 


Field  Environment 
(e.g.,  heat,  humidity,  mud) 


Training  Base  Station  (IBM  Computer)  Functions 

Set  ECS  Initial  -  The  Training  Base  Station  functions  were  appropriate  for 
Conditions  me  to  set  the  initial  conditions  -  Enter  Evaluator  Names. 

(Enter  Evaluator 
Names) 


Difficult  to  Use  Easy  to  Use 

If  the  Set  ECS  Initial  Conditions  Function  -  Enter  Evaluator  Names  was 
not  Easy  to  Use,  please  explain  how  it  should  be  different: 


Set  ECS  Initial  -  The  Training  Base  Station  functions  were  appropriate  for 
Conditions  me  to  set  the  initial  conditions  -  Enter  Soldier  Names. 

(Enter  Soldier 
Names) 


I _ 

Difficult  to  Use 


Easy  toUse 


If  the  Set  ECS  Initial  Conditions  Function  -  Enter  Soldier  Names  was 
not  Easy  to  Use,  please  explain  how  it  should  be  different: 


Training  Base  Station  Functions  (continued) 

Set  ECS  Initial  -  The  Training  Base  Station  functions  were  appropriate  for 
Conditions  me  to  set  the  initial  conditions  -  Select  Training  Guide. 

(Select  Training 


Difficult  to  Use 


Easy  to  Use 


If  the  Set  ECS  Initial  Conditions  Function  -  Select  Training  Guide  was 
not  Easy  to  Use,  please  explain  how  it  should  be  different: 


Diagnose  ECS  - 
Status 


The  Training  Base  Station  functions  were  appropriate  for 
me  to  diagnose  the  ECS  operational  status. 


Difficult  to  Use 


Easy  to  Use 


If  the  Diagnose  ECS  Status  Function  was  not  Easy  to  Use,  please  explain 
how  it  should  be  different: 


‘‘vTO 


s.v.yy 


Training  Base  Station  Functions  (continued) 

Transfer  ECS  -  The  Training  Base  Station  functions  were  appropriate  for 
Conditions  to  me  to  transfer  the  initial  conditions  to  the  Electronic 

Clipboard  Clipboard. 


Difficult  to  Use  Easy  to  Use 

If  the  Transfer  ECS  Initial  Conditions  to  Clipboard  Function  was  not 
Easy  to  Use,  please  explain  how  it  should.be  different: 


Upload  -  The  Training  Base  Station  functions  were  appropriate  for 

Training  Data  me  to  upload  scores  from  the  Clipboard. 


Difficult  to  Use 


Easy  to  Use 


If  the  Upload  Training  Data  Function  was  not  Easy  to  Use,  please  explain 
how  it  should  be  different: 


Training  Base  Station  Functions  (continued) 

Display  of  -  The  displayed  presentation  of  summary  score  reports  was 

Summary  Reports  appropriate  for  me  to  do  my  job. 

I _ I _ I _ I _ I _ I 

Difficult  to  Use  Easy  to  Use 

If  the  Display  of  Summary  Data  was  not  Easy  to  Use,  please  explain  how  it 
should  be  different: 


Entry  Correction/  -  The  Training  Base  Station  functions  were  appropriate  for 
Cancellation  me  to  correct  or  cancel  any  entries  that  I  made. 

I _ I _ I _ I _ I _ I 

Too  Complicated  Just  Right  Very  Unclear 


If  the  Entry  Correction/CanceUation  was  not  Just  Right,  please  explain  how 
it  should  be  different: 


Training  Base  Station  Functions  (continued) 

ECS  •  TBS  -  The  connection  between  the  Electronic  Clipboard  and  the 
Connection  Training  Base  Station  was  appropriate  for  me  to  successfully 

connect  the  two  devices. 


Too  Complicated  Just  Right  Very  Unclear 

If  the  ECS  -  TBS  Connection  was  not  Just  Right,  please  explain  how  it  should 
be  different: 


TBS  Menu  - 

Options 

(Useability) 


The  Training  Base  Station  menu  options  were  appropriate 
for  me  to  do  my  job. 


L 


Difficult  to  Use 


Easy  to  Use 


If  the  TBS  Menu  Options  were  not  Easy  to  Use,  please  explain  how  they 
should  be  different: 


25 


Training  Base  Station  Functions  (continued) 

TBS  Menu  -  The  Training  Base  Station  menu  options  were  appropriate 

Options  for  me  to  select  the  options  I  wanted. 

(Adequacy) 


Adequate  Not  Adequate 

If  the  TBS  Menu  Options  were  not  Adequate,  please  explain  how  they  should 
be  different: 


Open  Ended  TIMS  Evaluation  Questions 

If  you  could  change  the  Clipboard  in  any  way,  what  would  you  do? 


In  what  other  training  situations  do  you  think  the  Clipboard  could  be  used 
effectively? 


‘l*. 


s 
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Open  £nded  TIMS  Evaluation  Questions  (continued) 


In  what  other  evaluation  situations  do  you  think  the  Clipboard  could  be  used 
effectively? 


